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We describe a method for solving the joint problem of 
optimal routing and optimal bandwidth allocation in a 
network that supports plural subnetworks and plural com- 
munication services. Our method involves, for each source- 
destination pair communicating via a given subnetwork and 
a given class of service, determining a traffic rate to be 
offered to each of a set of permissible routes between that 
source and that destination, in the given subnetwork and 
service class. Our method further involves allocating a 
respective bandwidth to each link of each subnetwork. 
Significantly, the determinations of traffic rate to be offered, 
and the allocations of bandwidth to respective links of 
subnetworks, are performed in a mutually responsive man- 
ner. 
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METHOD FOR RESOURCE ALLOCATION 
AND ROUTING IN MULTI-SERVICE 
VIRTUAL PRIVATE NETWORKS 

FIELD OF THE INVENTION 

This invention relates to the design and management of 
communication networks of the kind that carry traffic cor- 
responding to more than one kind of service. More 
particularly, the invention relates to methods for distributing 
traffic among available routes and allocating bandwidth 
resources to such routes in communication networks of that 
kind. 

BACKGROUND OF THE INVENTION 

Networks are commonly used for exchanging or transfer- 
ring information between terminal communication devices 
such as computer terminals, telephones, facsimile machines, 
and computer file servers. FIG. 1 depicts an illustrative 
network. A typical network includes switching nodes such as 
nodes 10.1-10.8 of the figure, interconnected by links, such 
as links 20.1-20.10 of the figure. Each terminal communi- 
cation device (not shown in the figure) is generally associ- 
ated with one of the nodes. 

Each link has a certain capacity, usually characterized as 
a bandwidth for that link. Networks may carry information 
in various forms. However, the information is often format- 
ted into packets or cells, according to appropriate network- 
ing protocols. 

One such networking protocol is Asynchronous Transfer 
Mode (ATM). ATM is a networking protocol designed to 
efficiently support high-speed digital voice and data com- 
munications. 

When information is to be exchanged between a pair of 
terminal communication devices, the network establishes a 
path between the nodes associated with those devices. In the 
discussion that follows, the originating node will often be 
referred to as the "source", and the destination node will 
often be referred to as the "destination". The flow of 
information of a given service type s through such an 
established path will often be referred to as a communication 
"stream" from the source to the destination. 

We will use the term "physical path" to denote the source 
and destination for a given communication stream, together 
with intermediate nodes (if there are any), and the links that 
connect them. In networks of practical size and complexity/ 
there will often be a set of multiple physical paths, each able 
to carry the given stream. 

It should be noted that although a physical path may exist 
between a source S and a destination D, the full bandwidth 
of the links along that physical path may be unavailable to 
carry communication streams between S and D. That is, the 
network manager may specify a predetermined bandwidth 
for streams between S and D along each of the possible 
physical paths. The specified bandwidth may equal the full 
bandwidth, or some quantity less than the full bandwidth, or 
no bandwidth at all. The last case, of course, is equivalent to 
denying certain physical paths between S and D. 

We use the term "logical path" or "virtual path" to denote 
a path between a given source and a given destination as 
defined by such bandwidth specifications. The use of these 
terms signifies that these paths are not determined purely by 
physical considerations, but instead that they are partly 
defined by parameters that can be specified and can be 
changed. 

It will be clear from the foregoing discussion that indi- 
vidual virtual paths will often take up less than the full 
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bandwidth physically available on the links that they occupy. 
Thus, it will often be the case that several virtual paths 
co-exist along part or all of a physical path. Stated another 
way, each link of the network may simultaneously support 

S several virtual paths. 

An important advantage of virtual paths is that they can be 
arranged into subnetworks for serving the various users of 
the network, and that these subnetworks can be redimen- 
sioned (i.e., the bandwidth specifications along the various 

10 virtual paths can be revised) from time to time to serve 
changing traffic requirements. 

The problem of allocating bandwidth to all of the virtual 
paths in the network, according to source -destination pair, 
subnetwork, and class of service, is referred to as logical 
network design. One important element of logical network 

15 design is the problem of selecting a set of physical paths 
through the network having sufficient capacity for carrying 
the estimated traffic of a communication stream. The process 
that carries out solutions to this problem may take into 
account such factors as the network topology, the currently 

20 available buffer space at the nodes, and the currently avail- 
able link capacities. 

Significantly, the network operator may have quality-of- 
service commitments with its customers, such as guaranteed 
bandwidth or maximum cell-loss probability. The path- 

25 selection process may also take into account such commit- 
ments. 

As noted, certain networks, particularly high-speed 
networks, may support the networking protocol known as 
ATM. Such high-speed networks will typically carry mul- 

30 tiple services having different traffic characteristics, includ- 
ing both constant bit rate traffic and variable bit rate traffic. 
An important simplification in dealing with variable bit rate 
traffic in ATM networks is provided by the concept of 
effective bandwidth, which is described in detail in U.S. 

35 patent application Ser. No. 08/506160, "A Method for 
Admission Control and Routing by Allocating Network 
Resources in Network Nodes." 

Although buffering capacity plays an important role in 
ATM networks as a complement to bandwidth, this model 

40 subsumes the considerations related to buffering into the 
effective bandwidth. As a consequence, when variable bit 
rate traffic is characterized in terms of its effective 
bandwidth, any ATM network may be viewed (for call- 
handling purposes) as a multirate, circuit-switched, loss 

45 network in which the description of each service includes a 
characteristic bandwidth requirement (or rate) for each link 
that carries the service. 

In a simple circuit-switched loss network model that is 
useful for our purposes, each service route (or "virtual path") 

50 is treated as a communication channel spanning a relatively 
large bandwidth, and that can be subdivided into many 
smaller-bandwidth subchannels. Many calls are multiplexed 
on the virtual path. Each call occupies one or more of the 
subchannels. For our model, it is convenient to assume that 

55 this occupation is exclusive throughout the duration of the 
call A subchannel carrying a call is referred to as a 
"switched virtual circuit" or "switched virtual connection." 
Bits are assumed to be emitted by each source at a constant 
rate (corresponding to the effective bandwidth for the per- 

60 tinent service). Those familiar with Internet Protocol (IP) 
communications will appreciate that the traffic entities 
known as "flow 1 ' and "connection" refer to the IP counter- 
part of the switched virtual circuit. (In regard to IP, the 
protocol known as RS VP is invoked for reserving resources 

65 in source-destination paths, and thus it brings the IP concept 
of "flows" nearer the ATM concept of switched virtual 
circuits.) 
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Even when the conveniences afforded by the effective over a larger portion of the network. Because of this, an 
bandwidth model are applied to network analysis, the prob- iterative method must be used to approach the optimal set of 
lems of bandwidth allocation and routing can be very offered traffic rates by successive approximations. Mitra et 
difficult. One source of difficulty is the fact that in contrast takes an approach based on well-known techniques of 
to permanent virtual circuits, virtual circuits arrive, hold S optimization, such as the method of steepest ascent. Accord- 
network resources for some time, and then depart. As a *g t0 . method > knowledge of the sensitivity ofW to each 
consequence, it is necessary to take into account the ran- ° f *o various parameters 9sr is used to formulate the step 
domness of call arrivals and call holding times. A second ^ m cach cu / rent approximation to the next approximation, 
source of difficulty is the large potential link capacity in ™ e f ^nsitmties are referred to as the network sensitive 
these networks, which may reach thousands, or even tens of 10 ' A . , . , . ~ m- c 4 

thousands, otcucuits in the near future. Yet a third source of I. . me . th to *Jfrt ^ 

i* • .u i * *• i u r j-*p * supphed (blocks 30, 35, 40) for the service classes s, the 

difficulty is the large potential number of different services ^^b^ 

that these networks may carry. TTus number is expected to demand - for cach *' servicc class bctwccn cach sourcc . 

reach several hundreds of services in the near future, and desU oation pair a, the route sets R(s,a) for respective 

even more thereafter. is streams ( SjCT ) ( ^ me capacity allocations C, on the respec- 

There are well-known techniques for analyzing single- tive links, 

rate circuit switched networks. With the help of such An initial set of values for the p sr is provided (block 45). 

techniques, bandwidth allocation and routing can be usefully Then, the route and bandwidth- allocation data are used to 

performed, at least in the simpler networks. However, when determine the loss probabilities and the network sensitivities 

multiple rates are introduced, the sources of difficulty men- 20 (block 50). 

tioned above, among others, conspire to reduce the compu- Then, the values of the p jr are incremented according, 

tational tractability of problems of practical interest. e g ^ to me me thod of steepest ascent to give an improved 

The co-inventors named herein have made a significant value of W (block 55). 

contribution to the design problem in a multirate circuit Then, in an exemplary method of measuring the approach 

switched network. This contribution involves a routing 25 to a local optimum, a test for convergence (block 60) is 

method. This method is described in D. Mitra, et al., "ATM carried out by comparing the current and previous values of 

Network Design and Optimization: A Multirate Loss Net- w. jf the change in W is small enough, the procedure is 

work Framework," IEEE/ACM Transactions on Networking declared to have converged, and the current values of Wand 

4 (August 1996) 531-543, referred to below as Mitra etal. ^ the parameters p sr are output as the optimum values. 

This method is also described in U.S. patent application Ser. Otherwise, the procedure is iterated, using the current values 

No. 08/554,502, filed on Nov. 7, 1995 under the tide 0 f Pjr ^ in pu t. 

"Method for Logical Network Design in Multi-Service An arriving caU of a given stream may be offered to any 

Networks/' and commonly assigned herewith. route m me set of permissible routes for that stream. These 

The Mitra et al. routing method will now be briefly 35 offers may be approximately distributed over the routes of 

discussed with reference to FIG. 2. the route set according to respective, predetermined prob- 

The objective of the routing method is to find the rates of ability values. That is, for each route, the probability that a 

traffic offered to each route between a source-destination randomly selected call will be offered to that route has a 

pair that will optimize the performance of the network. The predetermined value. The optimized values of the param- 

performance of the network can be characterized in various 40 eters p sr are used to determine these probabilities, 

ways. In an exemplary analysis, it is characterized by the The output from the procedure of Mitra et al. also includes 

long-term network revenue W. W is determined by the the loss probability B w of each service type s on each link 

revenue earned per carried call per unit time e jr , the traffic 1. Procedures are available to derive, from this information, 

intensity (also referred to as "traffic rate") and the loss a measure of the desired bandwidth utilization for each link 

probability L sr . The subscript s takes on a different value for 45 of each service route, and from that to make nominal 

each service type, and signifies that each service type has its allocations of bandwidth for each stream to each link of each 

own peculiar set of values for these parameters. (In the fields service route. 

of network design and management, the traffic characteris- i t ^ becoming increasingly important for network opera- 
tics and measures of the quality of service are sometimes tors to offer their customers guaranteed quality of service 
said to be determined by the pertinent "application," and the 50 ( e ,g^ through service-level agreements), particularly for 
term "service" is used somewhat more narrowly. Here, we services of different types (and thus, different data rates), 
use the term "service" in its general sense, in which it is One very attractive approach to this problem is to organize 
synonymous with "application/*) the network into virtual private networks (VPNs). Sufficient 
The subscript r takes on a different value for each route in bandwidth is allocated to the various VPNs, on the links that 
the set of permissible routes between a given source and 55 they occupy, to provide the quality of service that has been 
destination, and signifies that even within a given service negotiated. To maximize his revenues, the network operator 
type, each route has its own peculiar set of values. The traffic will need to make the bandwidth allocations in such a way 
intensity is defined as the mean arrival rate of calls offered that when the traffic of all of the customers is optimally 
to the given route multiplied by the mean holding period of routed over their respective VPNs, a weighted aggregate 
a call. (For modeling purposes, we have assumed that call &q measure of bandwidth carried over the network, such as the 
arrivals have a Poisson distribution.) The loss probability is network revenue W, is maximized. 

defined as that fraction of calls that are blocked. Blocking The problem that this poses contains complexities that 

occurs if any link in the route refuses the call (due to have not been addressed adequately, if at all, in earlier work, 

insufficient resources to carry the call on that link). That is, the partitioning of links among different VPNs 

The loss probabilities have a complicated dependence on 65 forces a coupling between the problem of optimal routing 

the offered traffic rates. Specifically, small changes in traffic within a VPN, and the problem of optimal bandwidth 

loading in one portion of the network can have repercussions allocation to each VPN. 



08/22/2004, EAST Version: 1.4.1 



US 6,331,986 Bl 

5 6 

As we have discussed, the method of Mitra et al. is As noted above in reference to block 50 of FIG. 2, one 

advantageously applied to the routing problem in isolated step in the (iterative) procedure is to determine the loss 

VPNs. However, this method alone does not provide a probabilities h sr and the network sensitivities SW/Sp^ 

solution to the larger issues of network design when network which are best understood as the respective derivatives of 

resources are allocated among multiple VPNs. 5 network revenue with respect to the offered load of service 

type s on route r. 

SUMMARY OF THE INVENTION The step represented in FIG. 2 as block 50 is shown in 

We have invented a method for solving the joint problem FIG - 3 35 madc U P of ^rce smaller steps, shown respectively 

of optimal routing and optimal bandwidth allocation in a as blocks 65 > 70 > ^ 7S - Block 65 takes ^ tne 

network that supports plural subnetworks and plural com- 10 predefined effective bandwidths d sh the predefined link 

munication services. In a broad aspect, our invention capacities C„ and the current estimate of the offered loads 

involves, for each source-destination pair communicating P«- The procedure to be carried out in block 65 involves 

via a given subnetwork and a given class of service, deter- iteratively finding a self-consistent solution of two systems 

mining a traffic rate to be offered to each of a set of of C0U P led equations (for each service class s and link 1). 

permissible routes between that source and that destination, 15 These equations, referred to as fixed-point equations, are 

in the given subnetwork and service class. Our invention discussed in greater detail below. One system of equations 

further involves allocating a respective bandwidth to each 15 for the ^ loss probabilities B x/ , discussed below. If the 

link of each subnetwork. Significantly, the determinations of calculation is carried out using exact techniques, the calcu- 

traffic rate to be offered, and the allocations of bandwidth to lation of B -/ has complexity 0(C), where C characterizes the 

respective links of subnetworks, are performed in a mutually 20 capacities. (As explained below, the use of asymptotic 

responsive manner. techniques can reduce this complexity to 0(1).) 

The quantities that may be calculated as a result of block 

BRIEF DESCRIPTION OF THE DRAWING 65 include the link loss probability of calls of service s 

. . ..... r -n » *• 011 unk l > toe l° ad Yw-r of service s added to link 1 by route 

FIG. 1 is a schematic diagram of an illustrative commu- 25 f ^ indepcndent minQmg (i.e., dropping of calls) by all 

mcation network other Unks - n tfae giyen ^ lhe derivative of me ^ 

FIG. 2 is a flow chart of the method of Mitra et al. for loss probability with respect to each of the total thinned 

solving the optimal routing problem in a multiservice net- loads Together with the predefined network revenue e fr 

w °rk- earned per carried call per unit time, these quantities are 

FIG. 3 is an expanded detail of the flowchart of FIG. 2, 30 input to block 70 of the figure. Also output by block 65 is the 
showing further steps involved in determining loss prob- route loss probability L sn which is the equilibrium prob- 
abilities and network sensitivities. ability that after admission to the network, a call of service 

FIG. 4 is an expanded detail of FIG. 3, illustrating fixed class s will be dropped by route r. (A route will drop a call 

point equations. if any of its links refuses the call.) 

FIG. 5 is an illustration of the Uniform Asymptotic 35 In block 70, a set of parameters c sl is calculated for each 

Approximation (UAA), which may be used in connection service class, where 1 ranges over the values 1=1, 2, , . . , L, 

with the procedures of FIG. 3. L & tne tota l number 0 f Jinks in the network. These 

FIG. 6 is a schematic diagram illustrating the interaction parameters, referred to as implied costs, reflect the effective 

between the UAA and the procedures of FIG. 3. loss of revenue occasioned when the carrying ^of calls of a 

t-i^. - • , - . , ljQ , . r - - . 40 given service class on a given link reduces the remaining 

FIG. 7 U a high- evel flow chart of a I»oce» for solving * ^ » ^ increase m , h * 

the joint problem of optimal routing and optimal bandwidth f ^ offered ^ ^ le3dty of ^ 

allocation according to the invention in certain embodi- . . ■ ? . , . . ~ /o3l T 3x 

6 calculation, using exact techniques, is 0(S L ). 

ments. 



The network sensitivities are readily derived from the 

FIG. 8 is a simplified block diagram of a typical network 45 1qs& probabiliticSj the earnings parameters, and the 

switch useful in the practice of the invention. impUed m% flS mdicaled m box 75 of me figure . 

FIG. 9 is a more detailed flow chart of the process of FIG. Turning now to FIG. 4, the two fixed point equations are 
7. represented there by blocks 80 and 85, respectively. In block 
FIG. 10 is an expanded detail of the flow chart of FIG. 9, 80, each of the link loss probabilities is expressed in terms 
illustrating the optimum routing problem in the context of a 50 of the corresponding link capacity and, for each service class 
multiservice network that supports plural subnetworks. s, s=l, 2, . . . , S (S is the total number of service classes), 
FIG. His an expanded detail of the flow chart of FIG, 10, the corresponding total thinned link load and link band- 
illustrating a process for computing linearized capacity width. In block 85, each total thinned link load is expressed 
costs. in terms of the respective link loss probabilities for all links 
FIG. 12 is an illustration of the Refined Uniform Asymp- 55 and all service classes. As noted, an iterative technique is 
totic Approximation (RUAA), which may be used in con- used to solve these two systems of coupled equations 
nection with the procedure of FIG. 11. self-consistently. 

FIG. 13 is a flowchart of the processes of FIG. 11, as Io ^ cascs of Practical importance, the link capacities 

modified to incorporate the RUAA. b *">™ 50 th u at " «^ not economically or numen- 

60 cally feasible to solve the fixed point equations and find the 

DETAILED DESCRIPTION implied costs by exact techniques. In such cases, as 

I. The Optimal Routing Problem According to Mitra et al. described in Mitra et al., an asymptotic approximation 

As an aide to the understanding of the present invention, referred to as the Uniform Asymptotic Approximation 

it is helpful to first review the broad aspects of the procedure (UAA) may be used to reduce the complexity of the calcu- 

described in Mitra et al, cited above. The problem addressed 65 lations. In fact, small links may be treated by exact tech- 

by the procedure of Mitra et al is here referred to as the niques while, concurrently, larger links are treated by the 

Optimum Routing Problem. UAA. We refer to such an approach as a "hybrid" approach. 
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FIG. 5 provides a statement of the UAAfor the link loss 
probability B xf , expressed in terms of the link capacities, 
total thinned link loads, and link bandwidths. The functions 
brf and B, referred to in the figure are defined in Mitra et al. 
and need not be described here in detail, except to note that 
B, is independent of service class s. This is important for 
reducing the complexity of the calculations of loss and 
implied cost, as discussed below. 

The second expression that appears in FIG. 5 is a conse- 
quence of the UAA. It is a starting point for obtaining the 
implied costs. It is also important for reducing the complex- 
ity of the calculations of loss and implied cost. The symbols 
s and t are both indices of service class. The function anl is 
defined in Mitra et al., and need not be discussed in detail 
here. 

It should be noted in this regard that although the UAA 
has been found useful in connection with the Optimum 
Routing Problem as addressed by Mitra et al., we have found 
it preferable to use a different asymptotic approximation 
when solving the Optimum Routing Problem in the context 
of the present invention. Our preferred approximation is 
referred to as the Refined Uniform Asymptotic Approxima- 
tion (RUAA). The RUAA is preferable because it is more 
accurate for calculating implied costs and sensitivities to 
link loss probabilities. We provide a detailed discussion of 
the RUAA below, after turning to the features of the present 
invention. 

As indicated in FIG. 6, when the UAA (box 90) is 
introduced into the calculations, economy is achieved in 
solving the fixed-point equations (box 95) because the 
complexity of calculating the link loss probabilities can be 
reduced to 0(1) (i.e., it remains bounded as C increases 
without bound), and the complexity of finding the implied 
costs (box 100) can be reduced to OfL 3 ). Thus, the com- 
plexity of finding the implied costs is made independent of 
the number of service classes. This is especially significant 
for large multirate networks, that serve customers with 
diverse needs and thus carry traffic in many different service 
classes. 

II. The Joint Problem of Optimal Routing and Resource 
Allocation — Introduction 

We now mm to the features of the present invention, in an 
exemplary embodiment. The physical infrastructure network 
supports a plurality of subnetworks. We will often use the 
term Virtual Private Network (VPN) to refer to these sub- 45 
networks. For example, we may suppose that four customers 
are served by the network of FIG. 1: East Corporation, 
Midwest Corporation, West Corporation, and South Corpo- 
ration. Each of these has its own VPN, but links of the 
underlying network are shared among the various VPNs. 
Each corporation has its own patterns of usage of these 
various links. For example, within each VPN, traffic might 
be heavier between nodes that correspond to corporate main 
offices, than between other pairs of nodes. For purposes of 
the illustrative embodiment that we describe below, West 
will have a main office at nodes 10.1 and 10.2 of FIG. 1, 
Midwest will have a main office at node 10.3, East at nodes 
10.4, 10.5, and 10.6, and South at nodes 10.7 and 10.8. (For 
the same illustrative purposes, any node that is not a main 
office of a corporation will be a branch office.) 

There are Q distinct VPNs, indexed in the following 
discussion by the symbol Q. A bandwidth capacity C/ n) is 
allocated to each link I of each VPN R. The objective of the 
inventive techniques is to design the VPNs (i.e., to estimate 
the values of C/ 0) ) such that, when the traffic of each 
customer is optimally routed over its VPN, a weighted 
aggregate measure of carried bandwidth over the network is 
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maximized. In the following discussion, this weighted 
aggregate measure is referred to as network revenue. It is 
important to note that each of the VPNs will typically be 
supporting traffic in plural classes of service, each indexed 
herein by the symbol s. 

Symbols used above in the description of the Optimal 
Routing Problem according to Mitra et al. will also be used 
in the following discussion. However, those quantities that 
are peculiar to a particular VPN will be superscripted with 
the VPN index, as in the case of link capacities mentioned 
above. 

The decision variables (whose values are optimized in 
order to achieve the objective stated above) are the fink 
capacities C/ n) and the traffic intensity p f / n) of each stream 
and VPN that is admitted to the network and offered to a 

route r belonging to the set fRJ^ of permissible routes for 
that stream. (The various routes are indexed by the symbol 
r-) 

We assume in the following discussion that there are a 
total of S service classes, with each call of class s requiring 
a bandwidth d st on link L 

Although we believe that our inventive technique will 
have broad applications in practical networks, it should be 
noted that the embodiment to be described here was formu- 
lated with reference to a model network having certain 
idealized properties. As noted above, we have assumed that 
a packetized network can be modeled as a circuit -switched 
loss network, and that all considerations relating to buffering 
can be subsumed into the effective bandwidth. Moreover, we 
assume that call arrivals for each stream are Poisson 
distributed, with a mean rate A^^. Admission of each 
arriving call of stream (s, a) (CJ) , is assumed to be controlled 
by an independent Bernoulli trial. Thus, the outcome of such 
a trial determines whether the call is offered to VPN Q, and 

if so, to which route in the route set RjJ^ it is offered. If the 
call is accepted, the bandwidth on each link is assumed to be 
simultaneously held for the duration of the call. An arriving 
call offered to a given route may still be blocked if any link 
on the route has insufficient bandwidth. The holding periods 
of calls in stream (s,a) (£J > are assumed to be independent of 
earlier arrival and holding times, and to have a mean value 

The call arrivals of each stream (s, o) ( ; , for each route r 
in the permissible route set (Rg^ of that stream, are also 
assumed to be Poisson distributed, with a mean call-arrival 
rate X ffr (n) . The sum of the mean route call -arrival rates over 
the entire route set can be no greater than the mean call- 
arrival rate A JCT (n) for the entire stream. The service-route 
traffic intensity p jr (£i) is defined as Kf^fPso^, and the 
stream traffic intensity pjj^ is defined as A J( 



(«> (ft) 



We define an admission control parameter p JO (n) as 
(l/Ajo^ times the sum, over the entire route set, of the 
mean route call-arrival rates X sr ^ a \ This quantity expresses 
the Bernoulli probability that a given incoming call of the 
pertinent stream will be offered to the network. Similarly, the 
probability that a given call of service types will be offered 
to a route r is given by X^Va^"*. These admission 
probabilities and routing probabilities are obtained from the 
results of the inventive procedure. 

III. The Joint Problem of Optimal Routing and Resource 
Allocation — Overview 

The total network revenue W is the sum, over all of the 
VPNs, of the respective subnetwork revenues W (Q >. The 
subnetwork revenue is evaluated from the service-route 
traffic intensities p xr (Q \ the equilibrium probability LJ& of 
loss on each service route of a call after admission to the 
network, and the service-route earnings parameter e xr (0) . 
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The carried load of a given service route is given by p Jr (n) to as a route controller, and a router 123 in mutual commu- 

(l-L x / n) ). The product of this expression times the pertinent nication over a high-speed Ethernet link. Route controller 

earnings parameter gives the long-run average revenue for 122 is responsible, inter alia, for selection of routes and, 

the service route. The long-run average revenue for the through a software module 124 referred to as an element 

subnetwork is obtained by summing this expression over all 5 manager, for element management functions. The router 

streams (s, o) in the subnetwork, and over all routes in the inchides a respective interface card 125.1 for each of ingress 

route set for each stream ^nks 126.1, and a respective interface card 1252 for each of 

The objective of the joint problem addressed by our links ^^ Each of the egress interface cards 1252 

invention is to find the link capacities c,«» and the service- m ^ de , s a ^ U > f^J* mQm ?% 127 ' ™ C £ aig * ° f 
4 t & . , ray tU * • • ( l * + i t scheduling of packets. Memory 127 may be shared m 
route traffic intensities pj > mat maximize the tota net- 10 common ^ ^ 12 ' 2 (In ^ mslance s, each 
work revenue, subject to certain constraints One constraint of ^ cafds faas [{s Qwn r v espective memor y.) 
is that no service-route traffic intensity may have a negative Elemem manager 124 periodically reads the headers on 
value. A second constraint is that for all VPNs and all me incommg packets (or cells), and thereby collects infor- 
streams, the sum of the service-route traffic intensities oyer madon on source-to-destination traffic and utilizations of 
the pertinent route set must not exceed the corresponding 15 outgoing links. Significantly, the headers can also include 
stream traffic intensity p^"*. A third constraint is that the information identifying service class and VPN. A data base 
total capacity of each link must be perfectly allocated (i.e., (not shown), referred to as the Management Information 
neither under- nor over-allocated) among the VPNs. A fourth Base (MIB), typically resides at the element manager and 
constraint is that no subnetwork link capacity C/ n) may also at centralized network-management platform 128. The 
have a negative value. 20 relevant data (local or global, as appropriate) are stored in 
In at least some cases, it may be desirable to impose the MIB. Communication between each (local) element 
further constraints in order to assure a promised quality of manager 124 and the centralized network-management plat- 
service for each of some, or all, of the subnetworks. One form 128 is mediated by a protocol referred to as Simple 
such approach is to constrain each of the affected subnet- Network Management Protocol (SNMP). In addition to 
work revenues W (n) so that it cannot fall below a specified 25 transmitting information to the centralized network- 
value, management platform, each element manager also periodi- 
As shown in FIG. 7, the computational procedures of the cally downloads global network traffic information there - 
present invention are advantageously carried out by an from. This information permits the computational 
interplay between parent process 110 and child processes procedures described here to be carried out by, e.g., route 
115, of which there is one for each of the VPNs. The parent 30 controller 122. 

process includes the process of solving the resource alloca- The output of the computational procedures, namely the 

tion problem, given a set of service-route traffic intensities jointly optimized service-route traffic intensities and link 

corresponding to optimal routing for the current set of capacities, are readily applied by the route controller to 

subnetwork link capacities. Each of the child processes update the operating characteristics of the network by, e.g., 

includes the process of solving the optimal routing problem, 35 determining the parameters and weights in schedulers and 

given the current set of link capacities for its respective queuing engines. 

subnetwork. The entire procedure proceeds iteratively, with For example, each egress link from switch 121 typically 

each reallocation of link capacities followed by a new has its own scheduler, implemented by the cpu on the 

solution of the optimal routing problem for each of the corresponding interface card 125.2. Various scheduling 

subnetworks. The iterations cease when the total network 40 methods are available for carrying out the scheduling func- 

revenue is found to converge. tion. Weighted scheduling, such as weighted round-robin 

All of these computational procedures are readily carried scheduling, is particularly useful for enforcing service -level 

out by a digital computer under the control of an appropriate agreements. A weight may be described as the fraction of the 

computer program. Significantly, each of the child processes bandwidth to be given to a particular queue. Thus, the 

is carried out (in each iteration) independently of the others. 45 weights control the quality of service and resource allocation 

Therefore, the child processes are conveniently and advan- among VPNs, among services within VPNs, and among 

tageously carried out by a parallel processor. source -destination pairs within those service classes. The 

In practical applications for real-time network weights are computed by route controller 122 and down- 
management, traffic characteristics are measured in network loaded to router 123 for implementation in the respective 
120 and provided to parent process 110, exemplarily in the so schedulers. 

form of traffic matrices. (In practical networks, these mea- It should be noted in this regard that the computational 

surements are typically performed periodically, exemplarily procedures of FIG. 7 lead to a complete partitioning of the 

once per fifteen-minute period.) The entries in the traffic subnetworks. That is, link capacities are allocated strictly by 

matrix of a given subnetwork are the offered intensities of subnetwork, with no inherent sharing of allocated capacities 

stream traffic p J0 c "\ or the equivalent. In an exemplary 55 between any two subnetworks that happen to share the same 

matrix for a given subnetwork and service class, each source link. We have found, quite surprisingly, that the cost (in lost 

is represented by a respective row and each destination is capacity) of complete partitioning is relatively low. For 

represented by a respective column. example, our experimental simulations (some of which are 

The pertinent traffic measurements are, e.g., measuring described below) have shown, in a model network, that 

during the preceding time period and extrapolated to the 60 about 2% of the call blocking is attributable to the complete 

current time period. Alternatively, the pertinent description partitioning (by comparison with a network design based on 

of traffic characteristics is based, e.g., on historical data. complete sharing of network resources between 

The pertinent measurements are typically made by the subnetworks). In practical situations, this cost may be amply 

switches located at the various nodes of the network, and ofiset by the benefits of isolating the respective VPNs and 

collected by a centralized network-management platform. 65 protecting them from runaway borrowing of resources by 

For example, as shown in FIG. 8, a switch 121 typically other VPNs. Thus, there is greater assurance that the prom- 
includes a central processing unit (CPU) 122, often referred ised quality of service on each VPN can indeed be provided 
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Nevertheless, there may be cases in which, after the All of these various alternate and hybrid approaches are 

complete partitioning has taken place, it is advantageous to intended to lie within the scope and spirit of our invention, 

permit temporary borrowing of resources between VPNs, A significant simplification of the computational problem 

according, e.g., to current need on a call-by-call basis. (Such is achieved in our exemplary embodiment by linearizing the 

borrowing is exemplarily implemented by setting a sched- 5 dependence of subnetwork revenue upon link capacity. This 

uling weight known as a trunk reservation parameter.) procedure leads to linearized capacity costs w/ Q >. That is, 
Various procedures for such borrowing are well known to 

those skilled in the art. Specific examples are described, e.g., aty* n> _ ^ 

inD. Mitra and I. Ziedins, "Hierarchical Virtual Partitioning: ac} ftl SBW| 

algorithms for virtual private networking," Proa IEEE 10 

GLOBECOM 1997, pp. 1784-1791, and also in U.S. patent , . . 

application Ser. No. 08/649,502, commonly assigned It is a fortuitous feature of this hneanzed approach that the 
herewith, filed by D. Mitra and E. Ziedins under the tide rescmrcc allocation problem decomposes such that the real- 
method for Sharing Network Resources by Virtual Parti- locations to VPNs can be done for each link independently, 
turning " Typically, a control element at a network node is Moreover, it is possible to provide a closed-form solution to 
applies certain criteria in order to determine whether a the resulting linear programming problem, 
requested reallocation may be made from one VPN to More generally, nonlinear capacity costs could be used m 
another. The criteria may be based, for example, on the our approach as an alternative to linear costs, 
affected service class, the current load on the network link Approaches using nonlinear capacity costs are also 
(or other physical resource), and the current load on the 20 "tended to he within the scope and spirit of our invention, 
borrowing VPN. ( It should be noted that the decision to IV. Exemplary Embodiment of the Invention 
make such a reallocation may involve cooperation between TuraiQ g now to Fia 9 > « initialization procedure 129 
the source switch and one or more transit switches.) loading of initial link capacities C/ > as well as 
Thus, our invention is not limited to resource allocation values of ^ lowcr bound > ™* W k°™ d D ' \ 
and routing procedures having complete partitioning as an 25 on *f link-capacity increment n/ >, and thresholds H r and 
ultimate objective. Instead, the scope of our invention is W r that are used for testing the convergence of network 
intended to include procedures in which complete partition- revenue W. 

ing is an intermediate stage, followed by further adaptations. As shown in box 130 of the figure, the optimum routing 

As discussed above, Mitra et al. describes the use of an problem is solved (box 131) and the linearized capacity 

approximation referred to as the UAAfor determining loss 30 costs m now computed (box 132) for each of the subnet- 

probabilities and implied costs, and we similarly use an works Q > the current v ^f s of capacities. It 

approximation referred to as the RUAA in the exemplary sho * ld be noted * ihi f rc ^ each iteration of the 

embodiment to be described below. Both the UAA and the P rocess represented in box 130, the linearized capacity costs 

RUAA are examples of asymptotic techniques based on arc computed only after the optimum routing solution has 

stochastic models of the network traffic. Those skilled in the 35 converged. For each subnetwork, externally providedjnfor- 

art will appreciate that other models are also available, and matioa for to process includes the stream demands Pjo * \ 

may be appropriate in certain regimes of network the earnings parameters e^ 0 *, and the route sets RJ^. For 

complexity, network capacity, and traffic density. each subnetwork, the output of this process includes the 

One such class of models are the deterministic flow current service-route traffic intensities p jr (Q) , the current 

models, which treat network traffic, in some sense, as a 40 subnetwork revenue W (n) , and the linearized capacity cost 

continuous fluid. Deterministic flow models are much sim- w/ Q) for each link in the subnetwork. After this process has 

pier than the stochastic models that we discuss here, but been carried out for each of the subnetworks, the total 

when applied to achieve the objectives described here, they network revenue is calculated (box 135) and a convergence 

tend to pose relatively large linear programming problems. test is performed (box 140). In our current implementation, 

These models may be especially advantageous for treating 45 two conditions must be satisfied for convergence to be 

large networks, such as router-based networks of 500 nodes declared (i.e., for the logical variable "NOT_ 

or more. Deterministic flow models may be applied to the CONVERGED" to be set to FALSE): the fractional increase 

resource-allocation procedure, or to the routing procedure, in W must not exceed and the hypercube defined by all 

or to both. the current n/ n) and n/ n) must be bounded by This 

In fact, various hybrid approaches are possible, and 50 hypercube is defined as the set of all L-dimensional real- 

indeed may be advantageous when applied to certain prob- valued vectors, each of whose components x, (for each value 

lems. In one hybrid approach, mentioned above, the solver of 1) is no less than n/ n) and no greater than n/ 0) . This 

of the optimal routing problem applies a stochastic model, hypercube is bounded by H r if the maximum over all link 

but to estimate the link-loss probabilities and their partial indices 1 of the difference n, (£1) -n/ n) is no greater than H r , 

derivatives with respect to offered traffic, it treats small- 55 In case of convergence, the entire procedure ends, 

capacity links by exact techniques, and large-capacity links Otherwise, the linearized problem of optimal capacity real- 

by asymptotic techniques. In another hybrid approach, locations 145 is solved for all of the^ubnetworks, using the 

solver of the optimum routing problem applies asymptotic current values of the lower bounds n/ n \ the upper bounds 

techniques, but it uses the UAA during the early iterations n/ n) , the link capacities C/ n) , and the linearized capacity 

and switches to the RUAA for the final stages of conver- 60 costs w/ Q) for all of the subnetworks. In our current 

gence to maximal network revenue W. Thus, computational implementation, and as shown in FIG. 9, a separate iteration 

complexity is reduced for most iterations, but convergence of process 145 is performed for each of the respective links, 

is made more accurate in the last few iterations. In The ability to make this link-by -link decomposition 

approaches hybridized at a higher level, deterministic flow depends, in part, on the constraints that are imposed on the 

models are used for solving the optimal routing problem, but 65 revenue-maximization problem. Currently, our constraints 

techniques based on a stochastic model are used to solve the are only that total link capacity is conserved, and that each 

resource allocation problem, or vice versa. new capacity allocation must be non-negative and must not 
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individually exceed any link capacity. As noted above, other 
constraints may be added in order, e.g., to assure a specified 
minimum quality of service in one or more subnetworks. For 
example, such a constraint may specify that respective 
bandwidth carried by one or more subnetworks, in one or 
more classes of service, may not fall below a specified level 
(such as a specified fraction of total traffic offered to the 
pertinent subnetwork in the pertinent class or classes of 
service). This specified level may apply to total subnetwork 
bandwidth carried in the pertinent service class or classes, or 
it may even apply to bandwidth carried between one or more 
specified source-destination pairs. In such cases, the decom- 
position will not typically be available. Thus, instead of 
iterating process 145 for respective links, the process 145 
will solve a problem involving all of the links concurrently, 
such as a linear programming problem, or a non-linear joint 
optimization problem. 

As shown in box 150, process 145 results in new values 
for each of the link capacities C, (Q) , which are incremented 
by subtracting respective increments n/ n) (which are 
bounded, as discussed above). 

After process 145 has been executed for all of the links 
and the link capacities have_been updated, the lower bounds 
n/ n) and the upper bounds n/ n> are also updated (box 155), 
as discussed in greater detail below. The new values of the 
link capacities are then handed to process 130 for a further 
iteration. 

FIG. 10 is an expanded version of box 131 of FIG. 9, 
representing the process for solving the optimum routing 
problem. As indicated in FIG. 10, a further, externally 
provided input to this process is the set of effective band- 
widths d x /°^ of each service class and link, in the current 
subnetwork. As shown, the optimal service-route traffic 
intensities are found for maximizing revenue in the current 
subnetwork, subject to the constraints that: (i) the total traffic 
of a given stream offered to all permissible routes must not 
exceed the total traffic of that stream arriving at the network; 
and (ii) the traffic intensity p Jr (0) offered to any given route 
r must be non-negative. 

FIG. 11 is an expanded version of box 132 of FIG. 9, 
representing the process for computing the linearized capac- 
ity costs w/ n) (also referred to here as the network sensi- 
tivity to link capacity). As shown in FIG. 11, the first stage 
is to solve the fixed point equations (box 160). Included in 
the output of this stage are v^/" 1 , the total thinned load of 
service s offered to link 1 (for the current subnetwork £3). 
Also included in the output of this stage are the link loss 
probabilities (also referred to as blocking functions) B tf /°\ 
each expressing the loss probability of the respective service 
s on the respective link 1 (for the current subnetwork fi). The 
output of this stage is used in the next stage to compute the 
implied costs (box 165), and the implied costs, in turn, are 
used to compute the linearized capacity costs (box 170). 

As noted above, the RUAA is often advantageously used 
for obtaining the link loss probabilities. FIG. 12 provides, in 
box 175, a statement of this approximation. The second 
expression in FIG. 12 is a consequence of the RUAA. It is 
a starting point for obtaining the implied costs. It is also 
important for reducing the complexity of the calculations of 
loss and implied cost. The functions B w and G w and the 
coefficients A,/ 1 ** are defined below. 

The function L. (d/, y,, C ; ) for the link loss probability is 
well known to those skilled in the art. Methods for its 
evaluation are described in J. S. Kaufmann, "Blocking in a 
shared resource environment," IEEE Trans. Commun., vol. 
COM-29, pp. 1474-1481 (1981), and in J. W. Roberts, 
"Teletraffic models for the Telecom 1 integrated services 
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network," in Proc. Int. Teletraffic Congress-10, Session 1.1, 
paper #2. The symbol d, as an argument of the function 
L. stands for (dj,, d^, . . . , d j; ). The symbol v, as an 

5 argument of the function L. stands for (v w , v^, . . . , v xi ). 
FIG. 13 shows the same processes as FIG. 11, modified to 
incorporate the RUAA. As indicated by box 180, the process 
for solving the fixed point equations now includes, as output, 
coefficients A,/ 0 and a^/ 0 (defined below) for the current 

10 subnetwork. These coefficients are used in process 185 to 
compute the implied costs. Hie output of process 180 also 
includes for each link and service route (for the current 
subnetwork) the load v j/;r (Q) of service s added to link 1 after 
independent thinning by all other links in route r. This 

15 quantity is also used for computing the implied costs. The 
output of process 180 also includes the functions 
(defined below) for the current subnetwork. These functions 
are used in process 190 for computing the linearized capac- 

20 ity costs. 

As indicated in the figure, the RUAA is incorporated in 
both process 180 and process 185. Included in the output of 
process 185 are the functions (defined below) for the 
current subnetwork. These functions are used in process 190 

25 for computing the linearized capacity costs. Significantly, 
the complexity of process 185 can be made independent of 
the total number S of service classes when the RUAA is 
used. Although the actual complexity of this process 
depends upon the specific mathematical techniques used, a 

30 typical complexity for obtaining the implied costs when the 
RUAA is used is 0(27L 3 ). 

Mathematical Details 
1. Fixed Point Equations 
The VPN index Q is suppressed in this section. 
The fixed point equations (FPE) for the subnetwork are 
derived on the basis of the well known assumption of link 
independence. Each route r carrying traffic of service type s 
4 0 and using link 1 adds load v sl . r on link 1 which, by 
assumption, is Poisson, with a rate which is reduced by 
independent thinning by all other links in the route. By 
summing v sI;r over all routes r which use link 1 we obtain v sif 
the total thinned load of service s offered to link 1. Now let 
45 B sl denote the loss probability of service s on link 1 and let 

d;-{d w } x and y^- {v,,},. Then B jr L, (d,, v,, Q), which may 
be computed by tie Kaufman-Roberts recursion mentioned 
above, with complexity 0(C ; ). Alternatively, if C,»l and 
v,=0(C,), an approximation to may be computed by the 
50 RUAA, with complexity 0(1). In any case, we let § sl denote 
this mapping. The complementary part of the FPE gives the 
thinned offered load at each link as a function of the link loss 
probabilities. This mapping is denoted by ij). The complete 
subnetwork FPE are: 

55 

B.^M.MA • • - . S;l-U, ... X), 

v-*(B), (1) 

60 where 

vo{v/} and B={BJ. 

The numerical technique for solving the FPE is successive 
65 approximations, possibly damped. Finally, the loss probabil- 
ity of service s on route r is obtained from the solution 
of the FPE thus 
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integer. In particular, if we divide equation (7) by o, and let 

= 1 - f] (1 - *„); r e K <r\ < 2 > D ^°' ™ *** g radient 

2. Network Implied Costs 

The VPN index Q is suppressed in this section. 

One of the roles of the implied costs {C^}, as exhibited where 
in the following equations, is in determining the sensitivity 
of subnetwork revenue to offered loads: 10 



(3) 



These quantities may be given explicitly in terms of the 
unique positive z* which minimizes the strictly convex 
15 function 

The above highlights the importance of implied costs for 

traffic routing, where the decision variables are {p xr }. Here = y ^ _ 1} _ clogt ft > 0) (9) 

our focus is on capacity costs, where implied costs also *f 
feature importantly but, as we shall see, in a different way 
altogether. 20 

The calculation of {C„} requires the solution of a system ™ e F ( z ) 25 w f u « z * are common to both the UAA 

of SL linear equations, and RUAA. The mimmizer z* of (9) is readily calculated 

by bisection. We give below the expressions for cr > which 
are needed later. 



25 
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and 
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where {vj are obtained from the solution of the FPE (1), 
and 

4. RUAA in Implied Costs 

".i md *" 35 We consider the implication of the RUAA for the FPE (1) 

and the implied cost equations (4). The RUAA may be used, 
as previously noted, for the evaluation of the function in $ sI 
3. Single Link Asymptotics m (i). Also, from (7), reintroducing the link index 1, we have 

The VPN index Q is suppressed in this section. Symbols 
introduced in this section are defined urTable A. 4 o 3 (121 

Before proceeding to subnetworks, we focus on the use of L„(f*,, v,, Q - 4/) - L, {d r ^ Q) - £ A%a$, 

the RUAA for a single link, and just in this subsection we i=1 
suppress the link index 1. It is assumed that the link capacity 

C is large, and that the traffic intensities {v,} are also large, whcrCj from ^ a ri (0 -e/ 0 (d rf )/(l-B rf ). From (4), if the 
0(C). The RUAA to the loss probability for service s on 45 RUAA (12) is applicable for all links of the subnetwork, it 
the link takes the form follows that the implied costs are given asymptotically by 



(6) 



(13) 



50 



where B w i-1,2,3, does not depend on s. In the next two where 
subsections we also need an approximation to the change in 

the loss probabilities due to a change in the capacity. If ^ _ ( * \ ( 14 ) 

n=0(l) is a positive or negative integer, then the RUAA has 55 = / /, A * Vtl -\ Ctr ~ 2u zL a *» \ 

the form TT^^ \ isH ^' } 



3 (7) 

L^d. ju C - n) - U W. O ~ - 

h_ EB"fl»(ii)J 

Now G (0 (n) is an analytic function of n, and 6 C0 (0)-0, 

i-1,2,3. Although R", (d, v, C)=L. (d, v, |CD if C is not 
an integer, we use the approximation (7) when □ is not an 



Equation (14) is a complete system of equations in {|/°}. 

60 The parameters {A,/ 0 }, {v sI . r } and {a jJt w } are all obtained 
from the solution of the FPE (1). The complexity of solving 
the system of 3L linear equations (14) is 0(27 L 3 ), whereas 
the complexity of solving the system of SL linear equations 
(4) directly for the implied costs is 0(S 3 L 3 ). Independence 

65 from the number of services S constitutes a major gain if S 
is large. In any case, the approximation in (12), if applicable 
for link 1, may be used in (4). 
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5. Revenue Sensitivity to Link Capacity 
The VPN index Q is suppressed in this section. 
We consider the implication of the RUAA for the sensi- 
tivity of the subnetwork revenue to the link capacities. With 
the smooth continuation of L/d, y, C) for non-integer values 
of C, as previously discussed, we have been able to show 
that the corresponding gradient of the subnetwork revenue 
with respect to the capacity of link 1 is 



aw V V dL ° ( V ) 



(15) 10 



Hence, from equations (8), (13), and (14), we obtain 



(16) 



18 

-continued 



l"i n, l:Z"! m = 0 - v ' 

n 



The second constraint merely reflects the requirements that 
the new capacity allocations are nonnegative and cannot 
individually exceed any link capacity. 

There are two steps involved. The first, which is entirely 
intuitive given the linear structure of the cost function, and 
the separability of the constraints by links, establishes that 
(21) decomposes into L problems, one for each link. That is, 
with 



A W(a £n{ n) nS n, (/=l,2,... 1 L) 



(22) 



In summary, for large link capacities C, and traffic intensities to problem indexed by 1(1=1,2, .... L) is 
of the same order, 



aw_ (17) 



min Ah* t 23 ) 
n 



where 



3 . (18) In (23) the second constraint in (21) has been tightened to 

w/ = yV/^. 30 additionally ensure that the decision variables are not so 

/=i large that the approximation in (19) becomes invalid. This is 

done as follows: 

Here a/'\ i-1,2,3, are given by (10) and (11), with z* l{V-[min{c£V( comt )^] (24) 

replaced by z,* . Also, are obtained by solving the system 35 

of 3L linear equations in equation (14). n/°>=fmax (c/ Q) -C r (const)\/^w)J, (25) 

We call the quantities w, the linearized capacity costs. 

6. Mathematical Basis of Algorithm where a reasonable value of the constant (const) -0.5. It is 

Taking note of (17) and applying it to subnetwork Q important to ensure that the above bounds are integers so 

which has bandwidth C,™ allocated to it on each link 1, a 40 **t m the step to be described below the optimal solution, 

linearized extrapolation of the revenue for the subnetwork is and tos *e ncw ca P acit y ^cations, are also integral. Note 

* that 

£/ 0 >^0<H/°!> (26) 



l (19) 



45 in cases where C/ n) >0. 

We now proceed to examine the linear programming 
Here W/ fi) ,l-l,2, . . . , L> are the linearized capacity costs problem in (23) and give an explicit solution. Just here the 
given in (18), and integers n/ n) are candidates for modifying subscript 1 is redundant and to keep the notation simple it is 
the current capacity allocations. The latter quantities are suppressed. Also assume without loss of generality that 
required to be O(l) for (19) to apply. 50 

is defined as (C/°>, . . . , C <" C) ), and q c °> is defined s _ 

as (n,«», . . . , % <4 Also, C is defined as {C,^}. ■ ■ ^ < 27 > 

Note that because the network revenue is the sum of the ie> subne twork 1 is "cheapest", subnetwork U "most 
subnetwork revenues, 55 eX p en sive", and so on. Hence, the objective function 

decreases whenever we reduce n (n2) and increase n (C31) by 
* l m (20) an identical amount, provided Q 1 <Q 2 . Hence, the solution to 

w{Q-a)- w(Q & -^j/j^i rt < (23) consists of maximizing n (0 with small values of i and 

cui 1=1 minimizing n w with large values of i in a manner that the 

6 0 global constraint Zn (Q) =0 is satisfied. 
We proceed to select a such that W(£-n) is maximized. That A little thought shows that on account of (26) and (27), 
is ( there exists a unique subnetwork index I such that 

(21) . . . ^"W^i^U . . . +SP> (2S) 

n^Z Z H * /D>n ' n) 65 <o<&*>+ . . . +S'-04§W+1J£'«)* . . - +i (H >- (29) 

n=u=t 

Hence there exists n^, such that n^n^-cn^, and 
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3»>+n»+ . . . +r?'- l >+n«'W* 1 >+ . . . 4^-0 (30) 



TABLE A-coatimied 



Moreover, if all n (n) and d < ° ) are integral, then so is n w . 

Therefore, if n (/) , r7^£n w <r7^, is the unique solution to 

(30), then a solution to the linear program in (23) is s - (j + 1 j(j + 2)(?rK n < 0 



= An = /) 



TABLE A 



' MV2jtV 



^ ) (n) = £(j-l)(z r ) i ,n>0 

D-l 

*"(n) = ^ (j-l)Cj- 2X2')*, a >0 

-a 

^°(n) = -^(z*r i ,n<0 



fl«(0) = 0, i=I,2,3 



= n^{Cl =1 + 1.1 + 2 H). 10 ^(i) _ [3(3 _ i + z *) + (2B - 1 + z')(B (l > - b))(t l \d,) + 

B (2, [B0 t2) (d 1 ) + d,(c') < M + 



B< 3| [B6^ ) (d t ) + d t (d,-l)(z') d *] 

15 AJ2J = 3(2) (e _ j + 2 -)fl<i)(dJ + 2B (3 >d,(z*J d * 
Af^B^tB-l+z')^ 1 ^) 



F(z) = £ v l( z* - i) - Clogz M » = J v ' d > (d - " 10d > + 35 * " 5 W>* 



_ [l-3z' + 3(z-) 2 ] T(l-2z') 

C. = — ' + " » + 



_ V 2 .4. 0 " r)3 2V(1 " 

V "V I<I|U> Y l 1 vVVsgnU-O . 



25 



8[3V 2 Vj(l-z-) [-IFfr)?* 1 ,Z 



M = -Erfc[sgn(l-z')V^)"] + ^==. T 5I« 35I» Y , 5T\ IV 

E = 1 + 2V + 04W + 43^"m l + 6vi + 40V' Z " ! 

1 VVsgn(l-z') 



(I " z ') V-2F(z*) 



EXAMPLE 



j I s We briefly describe the results of our computational 

K = 2 + v,d *' 2 * " 1 experiments based on the methods described above. The 

* =l infrastructure network, shown in FIG. 1, has 8 nodes and 24 

35 0C3 links connecting 10 pairs of nodes in both directions. 
T = V v d?(d -Diz-f* There are 6 services, with effective bandwidth of individual 

S 1 calls being 16, 48, 64, 96, 384 and 640 Kbps, respectively, 

over all links. 

s ^ ^ 4Q There are 4 VPNs for the West, Mid West, East and South 

Y = }_j V *<W -6^ + ilXz*) Corporation, respectively. These are all national customers 

with traffic from every node to every node, for each of the 
5T 2 ^ 1\ services, with the traffic patterns for individual corporations 

B {1) = &|i + y Jv~"v2| + BE I exhibiting some preponderance of traffic to and from the 

I I \ J 45 nodes in the geographical areas corresponding to their 

3(2) = — names. The admissible routes are such that there are at most 

2V ^ 4 hops in the route. 

B (3) = - Table I gives a summary of the offered and carried 

bandwidth over the VPNs for the optimized design. Of 
„_! 50 particular importance is the absence of significant variability 

fl^Cn) = Yj fl > 0 in the blocking for the VPNs. 



TABLE I 

Summary Statis 

55 



VPN 


Summary Statistics for the Optimized Design 


Total 


West 


Mid West East 


South 


Offered 


18484 


17269 23167 


15758 


74679 


Bandwidth 










Carried 


17904 


16739 22530 


15230 


72400 


Bandwidth 










Blocking 


3.14 


3.07 2.75 


3.35 


3.05 


<%) 











_ n We compare the above results with those obtained by the 

fl t2, (n) = ^ (j + i)(z r p,n<0 65 Complete Sharing design in which the VPN structure is 

j- 1 eliminated and the aggregate traffic is optimally routed. The 

total carried bandwidth in the Complete Sharing solution is 
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73,795, which gives 1.2% blocking. Hence about 1.85% 
incremental blocking is due to the partitioning of resources 
among the VPNs. 

In Table II we consider link 20.2 (see FIG. 1) and give the 
capacity allocations to the VPNs in the optimized design. 

TABLE II 



Link 20.2 (capacity - 0C3) 



VPN 




West 


Mid West 


East 


South 


Relative Capacity Allocation 




32.9 


25.0 


24.9 


17.1 


to VPN (%) 












Relative Utilization in VPN 


1 


1.0 


0.3 


2.1 


1.8 


by service 














2 


2.7 


0.1 


6.2 


5.7 




3 


7.3 


0.6 


16.7 


14.7 




4 


4.6 


0.3 


12.4 


10.8 




5 


21.7 


23.2 


14.0 


38.7 




6 


62.7 


75.5 


48.6 


28.3 


Total Utilization in VPN (%) 




82.6 


79.2 


83.4 


80.5 



10 



Also given in the table are the utilization by each of the six 
services. Quite remarkable is the lack of significant vari- 
ability in the utilization of the link by the VPNs. 

Only 10 outer iteration were necessary to meet the con- 
vergence threshold of 10" 3 . Table III gives the total carried 
bandwidth after even numbers of iterations. Particularly 
noteworthy is the goodness of the initial design, which is a 
VPN design based on the Complete Sharing solution. 
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TABLE III 




Convergence in the Design Process 






0 


10 


Iterations 


(initial) 2' 4 6 8 


(converged) 


Carried 
Bandwidth 


72042 72207 72308 72359 72384 


72400 
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The invention claimed is: 

1. A method for allocating bandwidth capacity on links of 
a communication network that supports plural subnetworks 
and plural communication services, and in which sets of 
permissible routes are defined between respective source- 
destination pairs for each said subnetwork and service, 
wherein: 

the method comprises, in each of two or more iterations: 

(a) partitioning the bandwidth capacity of each of at 
least some links of the network among the subnet- 
works that share that fink; and 

(b) in each of two or more subnetworks, for each 
service and for each source-destination pair served 
thereby, determining a traffic rate to be offered to 
each permissible route between said source- 
destination pair; 

at least one instance of (a) is based on an allocation of 
traffic rates resulting from a prior instance of (b); and 

at least one instance of (b) is based on a partition of link 
capacity among subnetworks resulting from a prior 
instance of (a). 

2. The method of claim 1, wherein (a) and (b) are carried 
out so as to maximize a weighted aggregate measure of 
bandwidth carried by the network, said measure to be 
referred to as network revenue. 

3. The method of claim 2, wherein the network revenue is 
maximized subject to a constraint that specifies a minimum 
quality of service in at least one subnetwork. 
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4. The method of claim 3, wherein the constraint specifies 
that a respective bandwidth carried by said at least one 
subnetwork in at least one class of service may not fall 
below a specified level. 

5. The method of claim 4, wherein said bandwidth is the 
total subnetwork bandwidth carried in said at least one class 
of service. 

6. The method of claim 4, wherein said bandwidth is the 
subnetwork bandwidth carried in said at least one class of 
service between at least one specified source-destination 
pair. 

7. The method of claim 4, wherein the specified level is 
a specified fraction of the total traffic offered to the pertinent 
subnetwork in at least one class of service. 

8. The method of claim 2, wherein (b) is carried out such 
that the bandwidth allocated to each link is completely 
partitioned between subnetworks. 

9. The method of claim 8, wherein (a) and (b) are carried 
out iteratively, such that each iteration of (a) is responsive to 
a prior allocation of bandwidth to links, and at least the 
second and subsequent iterations of (b) are each responsive 
to a prior determination of traffic rates to be offered. 

10. The method of claim 9, wherein in each iteration, (a) 
is carried out using a deterministic flow model of network 
traffic. 

11. The method of claim 9, wherein in each iteration, (b) 
is carried out using a deterministic flow model of network 
traffic. 

12. The method of claim 9, wherein in each iteration, (a) 
is carried out using a stochastic model of network traffic, and 
(b) is carried out using a deterministic flow model of 
network traffic. 

13. The method of claim 9, wherein in each iteration, (a) 
is carried out using a deterministic flow model of network 
traffic, and (b) is carried out using a stochastic model of 
network traffic. 

14. The method of claim 9, wherein, in each iteration, (a) 
is carried out independently for each subnetwork. 

15. The method of claim 14, wherein (a) is carried out 
independently for each subnetwork by carrying out respec- 
tive computations for each subnetwork in parallel. 

16. The method of claim 14, wherein, in each iteration, (b) 
is carried out such that bandwidth is reallocated indepen- 
dently to each link. 

17. The method of claim 14, wherein the network revenue 
is evaluated by summing respective subnetwork revenues 
from each of the subnetworks, and the method further 
comprises, in each iteration: 

(a) for each link of each subnetwork, calculating a capac- 
ity cost that expresses the sensitivity of the pertinent 
subnetwork revenue to the bandwidth allocated to said 
link for said subnetwork; and 

(b) using the capacity costs to extrapolate from a current 
subnetwork revenue based on a current set of said 
bandwidth allocations to a new subnetwork revenue 
based on a reallocation of said bandwidths. 

18. The method of claim 17, wherein each of the capacity 
costs is a linearized capacity cost. 

19. The method of claim 17, wherein: 

a respective penalty referred to as an implied cost is 

associated with each link of each subnetwork for each 

respective service; 
each implied cost acts to reduce an effective revenue per 

call of the pertinent subnetwork and service that is 

routed on the pertinent link; 
the implied costs reflect probabilities that calls will be lost 

due to insufficient bandwidth on the various links of 

each subnetwork; and 
step (c) comprises evaluating implied costs. 
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20. The method of claim 19, wherein a stochastic model 
is used to describe network traffic, and for at least some 
links, the evaluation of implied costs is carried out using the 
Refined Uniform Asymptotic Approximation to express loss 
probabilities on said links. 

21. The method of claim 19, wherein a stochastic model 
is used to describe network traffic, and for at least some 
links, the evaluation of implied costs is carried out using 
exact methods to express loss probabilities on said links. 

22. The method of claim 19, wherein the evaluation of 
implied costs is carried out using a deterministic flow model 
to express loss probabilities on said links. 

23. The method of claim 19, wherein a stochastic model 
is used to describe network traffic, the evaluation of implied 
costs is carried out using exact methods to express loss 
probabilities on at least some links having relatively low 
capacities, and an asymptotic approximation is used to 
express loss probabilities on at least some links having 
relatively high capacities. 

24. The method of claim 19, wherein a stochastic model 
is used to describe network traffic, during at least some 
relatively early iterations, the evaluation of implied costs is 
carried out using the Uniform Asymptotic Approximation to 
express loss probabilities on at least some links, and during 25 
at least some relatively late iterations, the evaluation of 
implied costs is carried out using the Refined Uniform 
Asymptotic Approximation to Approximation to express 
loss probabilities on at least some links. 

25. The method of claim 9, further comprising, after the 
last iteration of (b), temporarily reallocating bandwidth in at 
least one link from at least one subnetwork to at least one 
other subnetwork. 
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26. The method of claim 1, wherein: 

the method further comprises measuring traffic intensity, 
in at least one class of service, offered to at least one 
source-destination pair within at least one subnetwork; 

steps (a) and (b) are carried out responsively to said 
measuring step; and 

the method further comprises, responsively to (a) and (b), 
setting at least one operating parameter of a network 
element. 

27. The method of claim 26, wherein the step of setting at 
least one operating parameter of a network element com- 
prises setting a scheduling weight. 

28. The method of claim 1, wherein: 

the method further comprises measuring traffic intensity, 
in at least one class of service, offered to at least one 
source-destination pair within at least one subnetwork; 

steps (a) and (b) are carried out responsively to said 
measuring step; 

the method further comprises, after (a) and (b), making at 
least one further measurement traffic intensity; and 

the method further comprises, responsively to said at least 
one further measurement of traffic intensity, tempo- 
rarily reallocating bandwidth in at least one link from 
at least one subnetwork to at least one other subnet- 
work. 

29. The method of claim 28, wherein said step of tem- 
porarily reallocating bandwidth in at least one link com- 
prises setting a trunk reservation parameter. 
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